EP 0 755 157 A3 


( 19 ) 



EuropaM 


s Patentamt 


European Patent Office 


Office europ6en des brevets 



( 11 ) 


EP 0 755 157 A3 


(12) EUROPEAN PATENT APPLICATION 

(88) Date of publication A3: (51) mt. Cl. 6 : H04N 7/58, H04N 7/32, 

17.09.1997 Bulletin 1997/38 H 04 N 7/50, G1 1 B 27/036 

(43) Date of publication A2: 

22.01.1997 Bulletin 1997/04 

(21) Application number: 96109036.2 

(22) Date of filing: 05.06.1 996 


(84) Designated Contracting States: 

DE FRGBITNL 

(30) Priority: 21.07.1995 US 505581 
(71) Applicant: 

MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD. 
Kadoma-shi, Osaka 571 (JP) 


(72) Inventors: 

• Egawa, Ren 

Princeton, New Jersey 08540 (US) 

• Meyer, Edwin Robert 
Bensalem, PA 19020 (US) 

(74) Representative: Schwabe - Sandmair - Marx 
Stuntzstrasse 1 6 
81677 Munchen (DE) 


(54) Method of splicing MPEG encoded video 


(57) A method of splicing two compressed video 
signals which have been encoded according to the 
standard adopted by the Moving Picture Experts Group 
(MPEG) determines an amount of null information that 
is to be inserted between the two video signals in order 
to ensure that an input buffer of an MPEG decoder does 
not overflow after receiving the spliced video signals. 
The method allows a splice to occur after any access 
unit (picture) in the first compressed video signal. The 
amount of null information is determined from the data 


rates of the first and second compressed video signals 
and the amount data which is provided to the buffer 
before the data is retrieved from the buffer for both the 
first and second video signals. The video signals are 
spliced by inserting the null information, as sequence 
stuffing bits into a buffer immediately after the selected 
picture in the first video signal. The second video signal 
is transmitted to the buffer immediately after these stuff- 
ing bits. 



BNSDOCID: <EP 075S157A3> 


Printed by Rank Xerox (UK) Business Services 
2.U 14/3 4 



EPO FORM 1501 0J.I2 (P04C01) 


EPO 755 157 A3 



European Patent 
Office 


EUROPEAN SEARCH REPORT 


Application Number 

EP 96 10 9036 



DOCUMENTS CONSIDERED TO BE RELEVANT 


Category 

Citatioa of document with indication, where appropriate, 
of relevant passages 

Relevant 
to daira 

CLASSIFICATION OF THE 
APPLICATION (IntXL6) 

X < < 

* •* * 

CL < CL < < Q. 

EP 0 692 911 A (MATSUSHITA ELECTRIC IND CO 
LTD) 

* the whole document * 

* column 3, line 6 - line 23; claims 1-4 * 

EP 0 676 756 A (SONY CORP) 11 October 1995 

* page 14, line 27 - line 51; figure 20 * 

& WO 95 08823 A 

EP 0 626 770 A (SONY CORP) 30 November 
1994 

* page 7, column 9, line 27 - column 10, 
line 38; figures 2,3 * 

SMPTE JOURNAL, 

vol . 104, no. 12, 1 December 1995, 
pages 788-802, XP000543847 
WEISS S M: "SWITCHING FACILITIES IN 

MPEG-2: NEVASSARY BUT NOT SUFFICIENT" 

* page 791, column 2, line 6 - line 27 * 

* page 793, column 1, line 6 - column 2, 
line 7 * 

* page 798, column 1, line 45 - column 3, 
line 59 * 

l-ii, 

13-16 

12 

1-16 

1-16 

1-16 

1-16 

H04N7/58 

H04N7/32 

H04N7/50 

G11B27/036 

TECHNICAL FIELDS 
SEARCHED (latai) 

H04N 

G11B 

Tlie present search report has been drawn up for all claims 

Piw •( ueth Oa« of ik u«a £xm*mr 

THE HAGUE 8 July 1997 De Paepe, W 

CATEGORY OF CITED DOCUMENTS T : theory or principle underlying the invention 

v . E = earlier patent document, but publi^tc*J on, or 

X : particularly relevant if taken alone after the filing date 

Y : particularly relevant if combined with another D : document ated in the application 

document of the same category L : document cited for other reasons 

A : technological background 

O : non-wntten disclosure & : member of the same patent family, corresponding 

P : intermediate document document 


8NSOOCIO: <EP 075S1S7A3> 



EP 0 755 157 A2 


(19) 




I! 


(12) 


(43) Date of publication: 

22.01.1997 Bulletin 1997/04 


Europai^Bs Patentamt 
European Patent Office 

Office europeen des brevets (11) EP 0 755 157 A2 

EUROPEAN PATENT APPLICATION 

(51) Intel 6 : H04N7/58 


(21) Application number: 96109036.2 

(22) Date of filing: 05.06.1996 


(84) Designated Contracting States: 

DE FR GB IT NL 

(30) Priority: 21.07.1995 US 505581 

(71 ) Applicant: MATSUSHITA ELECTRIC INDUSTRIAL 
CO., LTD. 

Kadoma-shi, Osaka 571 (JP) 


(72) Inventors: 

* Egawa, Ren 

Princeton, New Jersey 08540 (US) 

• Meyer, Edwin Robert 
Bensalem, PA 19020 (US) 

(74) Representative: Schwabe - Sandmair - Marx 
Stuntzstrasse 16 
81677 Munchen (DE) 


(54) Method of splicing MPEG encoded video 


(57) A method of splicing two compressed video 
signals which have been encoded according to the 
standard adopted by the Moving Picture Experts Group 
(MPEG) determines an amount of null information that 
is to be inserted between the two video signals in order 
to ensure that an input buffer of an MPEG decoder does 
not overflow after receiving the spliced video signals. 
The method allows a splice to occur after any access 
unit (picture) in the first compressed video signal. The 
amount of null information is determined from the data 


rates of the first and second compressed video signals 
and the amount data which is provided to the buffer 
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Description 

BACKGROUND OF THE INVENTION 

This invention concerns the processing of video 
signals encoded according to the standard adopted by 
the Moving Picture Experts Group (MPEG) and in par- 
ticular, a method of splicing an MPEG encoded data 
stream from a first source to an MPEG encoded data 
stream from a second source in a manner that ensures 
that no video data is lost when the combined image is 
reproduced. 

In the United States a standard has been proposed 
for digitally encoded high definition television signals. 
This standard is essentially the same as the MPEG-2 
standard, proposed by the Moving Picture Experts 
Group (MPEG) of the International Standards Organi- 
zation (ISO). This standard is described in a draft inter- 
national standard (DIS) publication entitled "Information 
Technology - Generic Coding of Moving Pictures and 
Associated Audio, Recommendation H.262" which is 
available from the ISO and which is hereby incorporated 
by reference for its teaching on the MPEG-2 systems 
standard "ISO/IEC 13818-1" and for its teachings on the 
MPEG-2 digital video encoding standard "ISO/IEC 
13818-2." 

The MPEG-2 standard is actually several different 
standards. In MPEG-2 several different profiles are 
defined, each corresponding to a different level of com- 
plexity of the encoded image. For each profile, different 
leveis are defined, each level corresponding to a differ- 
ent image resolution. One of the MPEG-2 "standards", 
known as Main Profile, Main Level is intended for 
encoding video signals conforming to existing television 
standards (i.e., NTSC and PAL). Another "standard", 
known as Main Profile, High Level is intended for encod- 
ing high-definition television images. Images encoded 
according to the Main Profile, High Level standard may 
have as many as 1,152 active lines per image frame 
and 1 ,920 pixels per line. It is this standard which is cur- 
rently being implemented as the United States HDTV 
standard. 

The MPEG-2 standard defines a complex syntax 
which contains a mixture of data and control informa- 
tion. Some of this control information is used to enable 
the signals having several different formats to be cov- 
ered by the standard. These formats define images, 
having differing numbers of picture elements (pixels) 
per line, differing numbers of lines per frame or field and 
differing numbers of frames or fields per second. In 
addition, the basic syntax of the MPEG-2 Main Profile 
defines the compressed MPEG-2 bit stream represent- 
ing a sequence of images in six layers, the sequence 
layer, the group pictures layer, the picture layer, the slice 
layer, the macro block layer, and the block layer. Each of 
these layers is introduced with control information and 
"stuffing" characters. The stuffing characters are 
inserted as needed to ensure that the data rate of the 
input data stream matches the rate at which pictures are 


displayed. Finally, other control information, also known 
as side information, (e.g. frame type, macroblock pat- 
tern, image motion vectors, coefficient zig-zag patterns 
and dequantization information) are interspersed 
5 throughout the coded bit stream. 

To effectively receive the digital images, a decoder 
must recognize the control portions, extract the neces- 
sary control information, and use the extracted data to 
process the video signal information. One piece of infor- 
io mation that is specified for each sequence layer is the 
video buffering verifier (VBV) buffer size. The VBV 
buffer size value specifies a number of bits of input data 
from the video sequence which must be stored in the 
input buffer of the decoder before the video sequence 
is may be decoded. If this number of bits is stored when 
the decoding operation begins, the input buffer will nei- 
ther become too full (overflow) or become empty 
(underflow) during the processing of the video 
sequence. 

20 The VBV buffer size value assumes that the input 
buffer is empty when the new sequence data is received 
or, at least that ail of the data in the input buffer will be 
removed before the contents of the input buffer reach 
the VBV buffer size value. To ensure that the VBV buffer 
25 size value does not cause the input buffer to overflow or 
underflow, it is preferred in the MPEG standard that any 
image splicing be done on a sequence boundary. 

This may be unacceptable, however, for video edi- 
tors who would like to switch from one scene to another 
30 after a particular picture has been displayed, irrespec- 
tive of whether that picture is at a sequence boundary. It 
may also be undesirable for broadcasters who wish to 
insert short program segments, for example station 
identification messages or advertisements into an 
35 HDTV program at arbitrary intervals. 

If splicing is not done at a sequence boundary then 
the data already resident in the input buffer may not be 
read from the buffer for processing until the buffer over- 
flows due to the new data from the inserted sequence. 
40 Alternatively, the data already resident in the input 
buffer may cause the decoder to prematurely indicate 
that the VBV buffer size criterion has been met. In this 
instance, the decoder may cause the input buffer to 
either overflow or underflow in the processing of the 
45 stored data. 

One method of solving this problem is proposed in 
the MPEG-2 standard DIS ISO/IEC 13818-1. By this 
method, certain access units, or pictures, in the data 
stream are marked with a SEAMLESS_SPLICE_FLAG. 
so This flag is used to mark safe locations for splicing an 
inserted sequence into the main sequence. As with the 
conventional splicing method, however, there are only a 
limited number of access units which satisfy the criteria 
to have this flag set. Furthermore, this splicing method 
55 limits the bit rate of the inserted stream to a certain level 
to meet its criteria and the inserted stream can not be a 
variable bit rate stream. 

Another method would be to define a format within 
the MPEG standard which allows for easy insertion and 
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provide a flag identifying this format. This method, how- 
ever, cannot handle incompatible formats and reduces 
the robustness of the MPEG standard. 

SUMMARY OF THE INVENTION 

The present invention is embodied in a method of 
processing a main MPEG encoded sequence to allow 
another sequence to be inserted at any picture bound- 
ary in the main sequence. According to the present 
invention, the main and inserted MPEG sequences are 
analyzed to determine a number of stuffing characters 
to insert in the data stream after the selected picture 
boundary and before the inserted sequence. The stuff- 
ing characters are ignored during the decoding opera- 
tion and allow the input buffer to be emptied to a level 
compatible with the inserted sequence. This number 
stuffing characters is determined from the data rates of 
the main and inserted sequences and the intervals 
between respective points in time corresponding to the 
start of the insertion and the next decoding time stamps 
in both the main and inserted sequences. The method 
inserts this number of stuffing characters after the 
selected splice point and then inserts the inserted 
sequence. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a data diagram which illustrates an 
insertion operation according to the present invention. 

Figures 2a and 2b are graphs of buffer fullness ver- 
sus time which are useful for explaining the operation of 
the present invention for fixed data rate MPEG-2 
encoded signals. 

Figures 3a and 3b are graphs of buffer fullness ver- 
sus time which are useful for explaining the operation of 
the present invention for variable data rate MPEG-2 
encoded signals. 

Figure 4 is a block diagram of circuitry suitable for 
use in an HDTV or MPEG-2 encoder and decoder which 
operates in accordance with the present invention. 

Figure 5 is a flow-chart diagram of a process which 
extracts parameter values from the main and inserted 
data stream. 

Figure 6 is a flow-chart diagram of a process which 
calculates a number of stuffing bits to be inserted from 
the extracted parameters. 

Figure 7 is a flow-chart diagram of an insertion 
process which inserts the number of stuffing bits calcu- 
lated in Figure 6 between the insertion point between in 
main sequence and the start of the inserted sequence. 

Figures 7a, 7b and 7c are graphs of buffer fullness 

versus lime wilier i are userui rui eApiaininy lire upeict- 
tion of the present invention when the variable k, in Fig- 
ure 7, has a value of one. 

DETAILED DESCRIPTION 

Figure 1 is a data diagram which illustrates the 


splicing operation. In this figures, a sequence 112 is 
inserted into a sequence 1 10 of the main HDTV signal 
to form a modified sequence 110'. As shown in Figure 1 , 
the main MPEG sequence 110 includes a sequence 
5 header 111, including a sequence start code, several 
access units (pictures) 1 16 and a sequence end code 
114. For the sake of simplicity, the group of pictures 
layer, which is between the sequence layer and the pic- 
ture layer is not shown in Figure 1 . The sequence to be 
io inserted, 112, also includes a sequence header 118, 
several access units 122 and a sequence end code 120. 

As defined in the MPEG standard, each sequence 
start code may be preceded by a string of zero-valued 
stuffing bits. In the MPEG standard, these bits represent 
is null information which is ignored by the decoder and are 
not written into the input buffer. The stuffing bits are 
used to ensure that the rate at which data is being sup- 
plied matches the rate at which images, derived from 
the data, are being displayed. While these stuffing bits 
20 are being received, the decoder is reading data from the 
input buffer for display. Thus, the amount of data held in 
the input buffer is reduced while the stuffing bits are 
being received. 

The combined sequences are shown as the modi- 
25 fied data stream 110’ of Figure 1. This data stream 
includes the sequence header 112, access units 1 16, a 
modified sequence header 1 18* access units 1 22 and a 
modified sequence end code 120*. The modified 
sequence start code 118' includes a sequence end 
30 code, which marks the end of the first part of the previ- 
ous main sequence, a number, NSTUFF, of stuffing bits, 
the sequence start code for the inserted sequence and 
the modified header for the inserted sequence. The bit- 
rate field of the original header for the inserted 
35 sequence is modified to be FFFF hexadecimal, indicat- 
ing a variable bit-rate sequence. This change is made 
only if the bit rate of the inserted sequence is different 
from that of the main sequence and the original inserted 
sequence was a constant bit-rate sequence. 

40 The method according to the present invention 
splices the start of one sequence to an arbitrary access 
unit in another sequence. The insertion of the sequence 
is completed only after the remaining part of the original 
sequence is spliced to the inserted sequence in the 
45 same way that the inserted sequence was spliced to the 
main sequence. Thus, the modified sequence header 
1 20* also includes a sequence end code, some stuffing 
bits and the modified sequence header for the main 
sequence. As with the modified sequence header 118’, 
so the sequence header 120* may have its bit-rate field 
changed to FFFF hexadecimal, indicating a variable bit- 
rate sequence, if the bit-rates of the main and inserted 

dcijuei uinei aiiu u ilic Cn iyiuai iViaiii ocrCjueTiCo uocu 

a constant bit-rate. 

55 The number of stuffing bits to be added is deter- 
mined by the method of the subject invention as 
described below with reference to Figures 5 and 6. The 
actual splicing operation is performed as described 
below with reference to Figures 2, 3 and 7. 
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Figures 2a and 2b are graphs of buffer fullness ver- 
sus time which illustrate the problem addressed by the 
present invention and the method by which the problem 
is solved. The solid line in Figure 2a shows part of the 
data stream for the main sequence, STREAM*!, while 5 
the solid line in Figure 2 b shows part of the data stream 
for the inserted sequence, STREAM2. In the exemplary 
embodiments of the invention, these data streams are 
bit-streams. Vertical transitions in these Figures indicate 
times at which data is being read from the buffer. It is a w 
convention in these types of diagrams that data is mod- 
eled as being read from the buffer instantaneously. Por- 
tions of the curve which have slanted lines indicate 
times at which data is being written into the buffer. The 
slope of these lines is proportional to the data rate of the 15 
data stream. 

The sequence STREAM 2 is to be inserted into the 
sequence STREAM 1 at time T\, immediately after 
access unit AU 1 . If the maximum input buffer size used 
by this example is 1000 then, if STREAM2 were 20 
inserted into STREAM 1 at time T t , then the buffer would 
overflow before the VBV buffer size (i.e. 500) for 
STREAM 2 had been reached. To prevent this from 
occurring, the method according to the present inven- 
tion inserts a number, NSTUFF, of stuffing bits between 25 
the AU 1 at time T f and the start of STREAM2. This is 
indicated by the dashed line in Figure 2a. Thus, after T,, 
instead of following the solid line, the buffer fullness for 
the MPEG bit stream follows the dashed line. The val- 
ues T (ast , DTSt, T next , DTSnext and DTS 2 are used, as 20 
described below to calculate the optimal value for 
NSTUFF While the stuffing bits are being inserted into 
the data stream, no new data is added to the buffer. 

New data, representing the inserted sequence is added 
when the slope of the dashed line is non-zero. 35 

It is noted, from Figures 2a and 2b that the rate at 
which data is provided by STREAM2 is less than the 
rate at which it is provided by STREAM 1 . Nonetheless, 
the first portion of the data of STREAM2, provided 
between the decoding time stamps DTS 1 to DTS nex t. is *o 
provided at the same rate as the sequence STREAM 1 
while the portion of STREAM2 which is provided after 
DT S next >s provided at the same rate as the original 
STREAM2, shown in Figure 2b. 

The graphs in Figures 2a and 2b illustrate the oper- 45 
ation of an exemplary embodiment of the inventive 
method when the sequences are encoded to have a 
fixed data rate. Figures 3a and 3 b illustrate the opera- 
tion of the method when the sequences are encoded to 
have a variable data rate. Items in these Figures which so 
correspond to items in Figures 2 a and 2 b are indicated 
by a prime mark ('). 

Figure 4 is a block diagram showing portions of an 
MPEG-2 or HDTV splicer and an MPEG-2 or HDTV 
decoder according to the present invention. The splicer 55 
includes a processor 412, which is coupled to receive 
the main and insert MPEG-2 HDTV signals, STREAM1 
and STREAM2, respectively. Each of these signals may 
be generated by a conventional HDTV encoder, such as 


is described in U.S. Patent No. 5.294,974 entitled, HIGH 
DEFINITION VIDEO ENCODING SYSTEM HAVING 
COLOR-SENSITIVE QUANTIZATION, which is hereby 
incorporated by reference for its teaching on HDTV 
encoders. Alternatively, these signals may be generated 
by an MPEG-2 encoder such as that described in 
ISO/IEC 13818-2 DIS, which is hereby incorporated by 
reference for its teachings on MPEG-2 encoders. 

The processor 412 is coupled to two buffers, BStest 
414 and BSdec 416. These buffers are preferably iden- 
tical. The buffer 414 is used to gather information from 
which the value NSTUFF is calculated. The actual splic- 
ing is performed using the buffer 416. The spliced signal 
is then transmitted from buffer 416 to the input buffer 
424 of the decoder 420. The decoder 420 also includes 
a processor 422 which retrieves data from the input 
buffer 424 in order to reproduce a moving image. 

Figure 5 is a flow-chart diagram which illustrates 
the process that controls the processor 412 in gathering 
information about the sequences STREAM 1 and 
STREAM2. This process begins at step 510, At step 
512, the processor initializes the buffer 414 by setting its 
write address pointer to the beginning of the buffer. At 
step 514, the processor sends STREAM 1 to the buffer 
at the data rate specified in the sequence header for 
STREAM1. It is noted that in the exemplary embodi- 
ment of the invention, both STREAM 1 and STREAM2 
are stored signals (e.g. recovered from digital video 
tape recorders). Thus, the signals may be accessed 
twice, once to determine splicing parameters and again 
to perform the actual splicing operation. For splicing two 
signals which are not stored but come, for example, 
from a satellite feed, a compensating delay (not shown) 
may be needed so that the splicing parameters may be 
determined before the actual splicing operation occurs. 

At step 514, the process determines if the insertion 
point, immediately after the picture represented by 
access unit AU1 has been stored into the buffer 414. 
Until AU1 is received, every access unit that is received, 
is removed from the buffer 414 at the time indicated by 
the decoding time stamp (DTS) field of the access unit. 
In other words, the data is removed from the buffer 414 
in the same sequence and with the same timing that it 
would be removed from an input buffer by a decoder, 
such as the decoder 420 shown in Figure 4. Even when 
each access unit is removed, the sequence STREAM 1 
continues to be provided at its proper data rate, as indi- 
cated by step 520, and control is transferred to step 516 
to determine if AU1 has been received. 

Once access unit AU1 has been received, control 
transfers to step 522 to determine values for T, ast and 
Niast- As shown in Figure 2a, T )ast is the difference in 
time between when AU1 was received and DTS1, the 
decoding time stamp for AU1 , the time at which AU1 is 
to be read from the input buffer for processing. N )ast is a 
number of bits transferred during the time interval T| ast . 
If STREAM1 is a constant bit-rate stream then N, ast is 
simply the T tast times the bit-rate. If STREAM1 is a vari- 
able bit-rate stream, however, N (ast may be any number 
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of bits less than the VBV buffer size. In a practical appli- 
cation where STREAM 1 is a variable bit-rate stream, 

N^ast may be T )ast times Rmax, where Rmax is the max- 
imum bit rate for the variable rate stream as long as N )ast 
* "Hast does not exceed the VBV buffer size. 5 

Next, at step 524, the process determines values 
for DTS next , T next , and N next . DTS next is the time to the 
next decoder time stamp after DTSl. In the MPEG 
standard, the decoder time stamps occur at regular 
intervals. Accordingly, DTS next may be determined by w 
calculating the difference between DTSl and the pre- 
ceding decoder time stamp and then adding this differ- 
ence to DTSl. Alternatively, this value may be 
determined by monitoring STREAM 1, extracting the 
DTS values from the access unit headers which follow is 
AU1 until a DTS other than DTSl is encountered. As 
shown in Figure 2a, T next is calculated by subtracting 
DTSl from DTS next . Likewise, N next , the number of bits 
of data which will be stored into the buffer during the 
time interval T next , is calculated by multiplying T next by 20 
the data rate, R1 , of STREAM 1 or by any rate which is 
less than or equal to the maximum data stream rate, 
Rmax[p.1], as defined in ISO/IEC 13818-1 and ISO/IEC 
13818-2. This data rate is available from the sequence 
header. If STREAMI is a variable rate data stream, as 25 
shown in Figure 3a, R1 is any data rate which is less 
than or equal to the maximum data rate for the data 
stream, R max [p.l], as defined in ISO/IEC 13818-1 and 
ISO/IEC 13818-2. 

At step 526, once these values have been deter- 30 
mined, the process again empties the buffer 414 and, at 
step 528, begins storing data from STREAM2 into buffer 
414. This data is stored into the buffer at the bit rate of 
STREAM2. As shown by the loop including step 532, 
the data is stored until a time corresponding to the 35 
decoder time stamp for the first access unit of 
STREAM2. This time is designated DTS2*. The amount 
of data stored in buffer 414 at DTS2* is designated as 
N nQW and this value is saved by the process at step 534. 

At step 536, the process determines whether 40 
STREAM2 is a fixed rate stream or a variable rate 
stream. If it is a variable rate stream, control is trans- 
ferred to step 538 to determine a value R2, representing 
the bit rate of STREAM2. In this instance, R2 is 
assigned the average value of the data rates over the 45 
time interval defined by DTS2*. It is contemplated that 
R2 may be any rate between this average data rate and 
the maximum data rate allowed for STREAM2, as deter- 
mined from its sequence header. At step 540, the proc- 
ess of collecting the parameters ends^ 50 

After the parameters have be oiB^t ermined. th e-.-af* 
processor 412 calculates the value N Sjp FF. This proc- Jvi 
ess starts at step 61 0 of Fiaure 6. At step 61 2. a tempo- 
rary variable K is set to zero. Next, step 614 calculates 
a value N splc , representing an amount of data which is ss 
to be written into the buffer having the data rate R2. 

In some splices, the amount of data represented by 
N new may not be able to be stored into the buffer, at the 
data rate R1, during the time interval T next . This situa- 


tion is handled by the loop which includes the steps 614, 
616 and 618. At step 614, a trial value for N sp i c is deter- 
mined as the product of K, T next and R2. At step 616, 
this trial value is compared to the quantity N new minus 
N,ast minus N next . 

If, however, N sp]c is greater than or equal to N new 
minus N )ast minus N next at step 616, then control is 
transferred to step 620 to determine the value of 
NSTUFF according to equation (1). 

NSTUFF = N las , + N next + N splc - N new (1 ) 

At step 622, the process ends. The inventors have 
determined that if this number of stuffing bits are added 
before the inserted sequence STREAM2, there will be 
no overflow of the input buffer when the data stream is 
received. 

After a suitable value of NSTUFF has been deter- 
mined, the processor 412 executes the process shown 
in Figure 7 to splice STREAM2 to STREAM 1. This proc- 
ess begins at step 710 by emptying the buffer BSdec, 
416, shown in Figure 4. At step 712, the processor 412 
sends data from STREAM1 into buffer 416 at the data 
rate of STREAMI. At step 714, the process determines 
if the last bit of AU1 has been written into buffer 416. If 
not, the process continues to provide data from 
STREAMI until AU1 has been stored as indicated by 
steps 716 and 718, removing any AU's prior to AU1 at 
intervals corresponding to their decoder time stamps 
(DTS’s). 

After AU1 has been stored, the process, at step 720 
removes any AU‘s in the buffer 416 which precede and 
include AU1 at the corresponding decoding time stamp 
of each AU. Starting from the time that AU1 has been 
stored, the process, at step 722 sends NSTUFF stuffing 
bits to buffer 416 at the bit rate R1. At step 724, the 
process determines if all NSTUFF of the stuffing bits 
were sent before the time DTS next . If not, control trans- 
fers to step 734 which determines if all NSTUFF of the 
stuffing bits were sent at the time DTS next . If this condi- 
tion is not met, step 736 is executed to send the remain- 
ing stuffing bits at rate R2 to buffer 416. 

If, however, at step 724, it was determined that the 
NSTUFF stuffing bits had been sent before DTS next , 
step 726 is executed which begins sending the initial 
part of STREAM2 to buffer 416 at bit-rate R1. At step 
728, the process determines if the variable K has a 
value of zero. If not, control transfers to step 738 in 
which data from STREAM2 is sent to buffer 416 at the 
bit-rate R2. Step 738 is also executed after step 734 if it 
is determined that NSTUFF bits have been sent at time 
DTS next and after step 736. After step 738, step 740 is 

pypmtoH uuhirh ramn\/oe Al I O frr>r*-» KnWar /lie ^ time 

corresponding to DTS2. 

Although the embodiments described above have 
used stuffing bits as the null information that is inserted 
between the spliced bit-streams, it is contemplated that 
other types of null information, for example, null pack- 
ets, could be used in appropriate circumstances. Alter- 
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natively codes that result "null time" may also be used. rate for STREAM2. in addition, larger values of K may 

These codes may, for example, temporarily suspend be desirable for some applications that need to maintain 


entering data into the VBV buffer. Whatever protocol is 
used, it is assumed that both the encoder and decoder 
operate according to the same protocol so that no infor- 
mation enters the VBV buffer during the defined time 
period. 

Figures 7a, 7b and 7c are graphs of buffer fullness 
versus time which show a splice in which K has a value 
of one. In Figure 7c, it is noted that between the time 
indicated by DTS next and the time indicated by DTS2, 
the bit rate is R2, the bit rate of STREAM2. if STREAM2 
is a constant bit-rate sequence, and it is the average bit- 
rate during T2* (shown in Figure 7b) if STREAM2 is a 
variable bit-rate sequence. 

If, however, at step 728, it is determined that the 
variable K is equal to zero, then at step 730, first access 
unit of STREAM2, AU2, is removed from buffer. After 
step 730 or step 740, step 732 is executed which contin- 
ues to provide data from STREAM2, at the bit-rate of 
STREAM2. This is the last step of the splicing process. 

As shown in Figure 4, data is removed from buffer 
416 and applied to the input buffer 424 of the decoder 
420. The buffer 424 ignores the stuffing bits, effectively 
delaying the application of the STREAM2 data until suf- 
ficient data has been removed from the input buffer 424, 
by the processor 422, to make room for the STREAM2 
data. 

The embodiments described above use the interval 
between successive decoding time stamps as the inter- 
val in which the amount of data in STREAM1 and 
STREAM2 are measured. Alternatively, other time inter- 
vals may be used. In one alternative embodiment, for 
example, the interval may be a frame interval and the 
number of bits provided in either STREAM 1 or 
STREAM2 may be the number of bits that are provided 
in a frame interval. For constant bit-rate streams, this 
number may be calculated as the bit-rate times the 
frame interval. For variable bit- rate streams, this number 
may be the maximum bit rate times the frame interval. In 
this alternative embodiment, N !ast may be the number of 
bits provided in STREAM1 during one frame interval; 
N next may be the number of bits provided in STREAM2 
during one frame interval, N sp | C may be determined as 
an integer, K, times N next . The value for N new in this 
embodiment is the same as in the previous embodi- 
ment. Using this alternative embodiment, the amount of 
null information (or null time), NSTUFF, that needs to be 
inserted into the bit-stream is defined by the equation 
(2). 

NSTUFF = N , ast + K * N next - N new (2) 

In equation (2), K is an integer number that satisfies 
the inequality K * N next *a N new - N lasl . In addition, the 
value of K may depend on the two data streams that are 
being spliced. For example, if the two data streams are 
constant bit-rate streams having different rates, a larger 
value for K may be desirable to maintain the constant bit 
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field parity consistency across the sequence boundary 
or that need longer time intervals to respond to changes 
5 in sequence header parameters between STREAM 1 
and STREAM2. One of ordinary skill in the art of editing 
MPEG bit-streams could readily determine suitable val- 
ues of K for a particular application. 

While the invention has been described in terms of 
10 an exemplary embodiment, it is contemplated that it 
may be practiced as described above in accordance 
with the following claims. 

Claims 

is 

1 . A method for spliciing first and second data streams 
representing respective first and second encoded 
signals at a selected boundary in the first data 
stream using a buffer, wherein each of the first and 

20 second data streams has a plurality of decoding 
time stamps representing times at which data is 
retrieved from the buffer, the method comprising the 
steps of: 

25 analyzing the first and second data streams to . 

determine an amount of null information to be 
inserted in the first data stream after the 
selected boundary, wherein The amount of null 
data is determined from the decoding time 
30 stamps in the first and second data streams; 

transmitting the first data stream, up to the 
selected boundary to the buffer; 

35 transmitting the determined amount of null 

information to the buffer; 

transmitting the second data stream to the 
buffer; and 

40 

retrieving the spliced data stream from the 
buffer. 

2. A method according to claim 1 , wherein the step of 
45 transmitting the second data stream to the buffer 

includes the steps of: 

transmitting a first portion of the second data 
stream to the buffer at the data rate of the first 
so data stream; and 

transmitting a second portion of the second 
data stream to the buffer at the data rate of the 
second data stream.. 

55 

3. A method according to claim 2, wherein the data 
streams are bit-streams encoded according to a 
standard developed by the motion-picture experts 
group (MPEG) and the amount of null information is 
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a number of stuffing bits to be inserted before a 
sequence start code of the second data stream.. 

4. A method according to claim 3, wherein the amount 

of null information is determined by the equation: s 

NSTUFF = N (ast+ N next + N splc .N new 

where N !ast represents a number of bits in 
the first data stream between the time that data fol- io 
lowing the selected boundary is stored into the 
buffer and the decoder time stamp at which the data 
following the selected boundary is retrieved from 
the memory: N next represents a number of bits of 
the first data stream that would be provided in an is 
interval between two decoder time stamps: N sptc 
represents a number of bits of the first bit-stream 
that would be provided in an integer number of 
intervals between decoder time stamps of the first 
data stream, the number of bits being greater than 20 
or equal to N new , the number of bits in the second 
data stream occurring before the first decoder time 
stamp of the second data stream.. 

5. A method according to claim 1 wherein the first and 25 

second data streams are constant bit-rate data 
streams and the step of analyzing the first and sec- 
ond data streams to determine the amount of null 
data is responsive to the respective bit-rates of the 
first and second data streams.. 30 

6. A method according to claim 1 wherein the first data 
stream is a constant bit-rate data stream and the 
second data stream is a variable bit-rate data 
stream having a maximum bit-rate and the step of 35 
analyzing the first and second data streams to 
determine the amount of null data is responsive to 

the bit-rate of the first data stream and to the maxi- 
mum bit-rate of the second data stream.. 

40 

7. A method according to claim 1 wherein the first data 
stream is a variable bit-rate data stream having a 
maximum bit-rate and the second data stream is a 
constant bit-rate data stream and the step of ana- 
lyzing the first and second data streams to deter- 45 
mine the amount of null data is responsive to the 
maximum bit-rate of the first data stream and to the 
bit-rate of the second data stream.. 

8. A method according to claim 1 wherein the first and so 
second data streams are variable bit-rate data 
streams, each having a respective maximum bit- 

rato anH tho etan r\i analuTinn fire* flnH earnnH 

data streams to determine the amount of null data 
is responsive to the respective maximum bit-rates ss 
of the first and second data streams.. 

9. A method according to claim 1 further including the 
steps of: 


analyzing the first data stream to determine if 
the first data stream includes a sequence end 
code immediately before the selected bound- 
ary: and 

transmitting the sequence end code to the 
buffer after the first data stream is transmitted 
to the buffer when the first data .stream does 
not include a sequence end code immediately 
before the selected boundary. 

10. A method according to claim 1 further including the 
steps of: 

analyzing the second data stream to determine 
if the. second data stream begins with a 
sequence start code; and 

transmitting the sequence start code to the 
buffer before the second stream is transmitted 
to the buffer when the second data stream 
does not begin with a sequence start code. 

1 1 . A method of splicing first and second data streams 
representing respective first and second encoded 
video signals at a selected boundary in the first 
data stream using a buffer, wherein each of the first 
and second video signals has a frame rate and 
wherein the second data stream has at least one a 
decoding time stamp representing a time at which 
data is retrieved from the buffer, the method com- 
prising the steps of: 

analyzing the first and second data streams to 
determine an amount of null information to be 
inserted in the first data stream after the 
selected boundary, wherein The amount of null 
data is determined from the frame rates of the 
first and second video signals and the decod- 
ing time stamp of the and second data stream; 

transmitting the first data stream, up to the 
selected boundary to the buffer; . 

transmitting the determined amount of null 
information to the buffer; 

transmitting the second data stream to the 
buffer; and 

retrieving the spliced data stream from the 
buffer. 

12. A method according to claim 11, wherein the 
amount of null information is determined by the 
equation: 

NSTUFF = N la8t + K * N nexl - N new 
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where N )ast represents a number of bits in 
the first data stream between the time that data fol- 
lowing the selected boundary is stored into the 
buffer during a frame interval of the first video sig- 
nal; Nnext represents a number of bits of the sec- 
ond data stream that would be provided in one 
frame interval of the second video signal; Nnew 
represents the number of bits in the second data 
stream occurring before the decoder time stamp of 
the second data stream and the value of K is deter- 
mined by the inequality K * N next ^ N n6W - N last . 

13. Apparatus for splicing first and second data 
streams representing respective first and second 
encoded signals at a selected boundary in the first 
data stream using a buffer, wherein each of the first 
and second data streams has a plurality of decod- 
ing time stamps representing times at which data is 
retrieved from the buffer, the method comprising the 
steps of: 

means for analyzing the first data stream to 
determine an amount of null information to be 
inserted in the first data stream after the 
selected boundary, wherein The amount of null 
data is determined from the decoding time 
stamps in the first and second data streams; 

means for transmitting the first data stream, up 
to the selected boundary to the buffer; 

means for transmitting the determined amount 
of null information to the buffer; 

means for transmitting the second data stream 
to the buffer; and 

means for retrieving the spliced data stream 
from the buffer. 

14. Apparatus according to claim 13, wherein the first 
data stream has a constant data rate and the 
means for transmitting the second data stream to 
the buffer includes: 

means for transmitting a first portion of the sec- 
ond data stream to the buffer at the data rate of 
the first data stream; and 

means for transmitting a second portion of the 
second data stream to the buffer at a data rate 
determined from the second data stream. 

15. Apparatus according to claim 13 further compris- 
ing: 

means for analyzing the first data stream to 
determine if the first data stream includes a 
sequence end code immediately before the 


selected boundary; and 

means for transmitting the sequence end code 
to the buffer after the first data stream is trans- 
5 mitted to the buffer when the first data stream 

does not include a sequence end code before 
the selected boundary. 

16. Apparatus according to claim 13 further compris- 
io ing: 

means for analyzing the second data stream to 
determine if the second data stream begins 
with a sequence start code; and 

15 

means for transmitting the sequence start code 
to the buffer before the second stream is trans- 
mitted to the buffer when the second data 
stream does not begin with a sequence start 
20 code. 
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